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(54) PANORAMIC IMAGE GENERATION DEVICE AND ITS METHOD 
(57)Abstract: 

PROBLEM TO BE SOLVED: To generate a panoramic image that has high 
picture quality and has no distortion caused at each connection part of the image 
by mapping plural images which are photographed by controlling the 



photographing direction of an image pickup means via a drive means onto a 
virtual spherical surface that is viewed from the rotational center of the drive 
means. 

SOLUTION: A computer 1 photographs a scenery 4 by controlling a pan tilter 
camera 3 located at a remote place and displays the image of a photographed 
screen 5 in an operation area 6A of a monitor 2. When a panoramic screen 
production instruction is produced via a panorama production button 6E, a pan 
tilter and a zoom lens are driven to each prescribed position to acquire each 
image. These acquired images are connected together to a virtual spherical 
surface defining a mobile axis of the pan tilter as an original point, and a plane 
image which is normalized in both latitude and longitude directions of the virtual 
spherical surface is shown in a panoramic image display area 6B. In such a way, 
a panoramic image is easily produced with no use of an expensive 
super-wide-angle lens nor fisheye lens. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not 
reflect the original precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 

CLAIMS 
[Claim(s)] 

[Claim 1] Panorama image generation equipment characterized by to consist of 



an image connection means is distorted by mapping to the virtual spherical 
surface seen from the bottom of its heart during the revolution of the 
above-mentioned driving means, and connect two or more images obtained by 
image pick-up means photo an image, the driving means which controls the 
bearing which the above-mentioned image pick-up means picturizes, and the 
above-mentioned image pick-up means that there is nothing. 
[Claim 2] Panorama image generation equipment characterized by mapping 
further the above-mentioned image mapped to the above-mentioned virtual 
spherical surface at a flat surface by normalizing at . the LAT of the 
above-mentioned virtual spherical surface, and LONG with the above-mentioned 
image connection means in claim 1 . 

[Claim 3] It is panorama image generation equipment characterized by having 
the device in which the above-mentioned driving means has play in a drive 
system in claim 1 , and amending the effect of backlash using the image acquired 
by driving the above-mentioned driving means in the same direction when 
connecting the image obtained from the above-mentioned image pick-up means. 
[Claim 4] Panorama image generation equipment characterized by connecting 
an image whenever it acquires the image of one sheet, displaying the result on a 
display means, and displaying the progress situation of image connection in 
claim 1 when connecting the image of two or more sheets with the 



above-mentioned image connection means. 

[Claim 5] Panorama image generation equipment characterized by making it 
display on the above-mentioned display means after not displaying the 
connection result on a display means whenever it acquires the image of one 
sheet in claim 1, when connecting the image of two or more sheets with the 
above-mentioned image connection means, but acquiring all images. 
[Claim 6] Panorama image generation equipment characterized by displaying 
the result on a display means when the image of two or more sheets is 
connected with the above-mentioned image connection means, the result is not 
displayed on a display means, but all images are captured and connection is 
completed in claim 1 although an image is serially connected whenever it 
acquires the image of one sheet. 

[Claim 7] Panorama image generation equipment characterized by picturizing 
the image of the movable range of the above-mentioned driving means with the 
above-mentioned image pick-up means, and generating a panorama image 
automatically in claim 1. 

[Claim 8] Panorama image generation equipment characterized by displaying 
the above-mentioned panorama image on the actuation field for showing the 
driving direction of the above-mentioned driving means further in claim 7. 
[Claim 9] Panorama image generation equipment characterized by moving the 



photographic subject of the location corresponding to one point of the specified 
arbitration to the position coordinate of the arbitration of the above-mentioned 
driving means by specifying one point of the arbitration of the above-mentioned 
actuation field in claim 8. 

[Claim 10] Panorama image generation equipment characterized by moving the 
photographic subject of the location corresponding to one point of the arbitration 
generated from the field of the above-mentioned arbitration to the position 
coordinate of the arbitration of the above-mentioned driving means by specifying 
the field of the arbitration of the above-mentioned actuation field in claim 8. 
[Claim 11] The panorama image generation method characterized by consisting 
of a step which is distorted by mapping to the virtual spherical surface seen from 
the bottom of its heart during the revolution of the above-mentioned location 
which carries out photography, and connects the step which photos an image, 
the step which controls the bearing which picturizes the above-mentioned image, 
and two or more images obtained by taking a photograph that there is nothing. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 



[0001] 

[Field of the Invention] this invention - high resolution and the panorama image 
of high quality — cheap equipment — and it is related with the panorama image 
generation equipment and the approach of acquiring easily. 
[0002] 

[Description of the Prior Art] When images were connected conventionally, the 
location (pixel) which an image to connect correlates was found out and the 
technique of piling up, after piling up simply, expanding / reducing the image 
connected to the image of one sheet or deforming was taken. 
[0003] 

[Problem(s) to be Solved by the Invention] However, amplification/cutback of an 
image, and deformation had the problem which brings about deterioration of 
image quality, such as degradation of the resolution of an image. Moreover, 
when two or more images were only connected, there was a problem which 
distortion generates. For example, when connecting the images often sheets as 
shown in drawing 21 A, the image of ten sheets obtained from image pick-up 
equipment maps the plane group vertical to the optical axis of a lens. For this 
reason, when having projected between the images which the photographic 
subject which exists in the depth direction connects according to the technique 
of only connecting the image of ten sheets, as shown in drawing 21 B, the 



problem which distortion produces was in the joining segment. 

[0004] Therefore, the object of this invention is high definition and is to offer the 

panorama image generation equipment and the approach of generating the 

panorama image which distortion does not produce in a joining segment. 

[0005] 

[Means for Solving the Problem] Invention according to claim 1 is panorama 
image generation equipment characterized by to consist of an image connection 
means is distorted by mapping to the virtual spherical surface seen from the 
bottom of its heart during the revolution of a driving means, and connect two or 
more images obtained by image pick-up means to photo an image, the driving 
means which controls the bearing which an image pick-up means picturizes, and 
the image pick-up means that there is nothing. 

[0006] Invention according to claim 11 is a panorama image generation method 
characterized by consisting of a step which is distorted by mapping to the virtual 
spherical surface seen from the bottom of its heart during the revolution of the 
location to photo, and connects the step which photos an image, the step which 
controls the bearing which picturizes an image, and two or more images 
obtained by taking a photograph that there is nothing. 

[0007] With an image pick-up means (for example, video camera), while 
changing into an electrical signal the image of the photographic subject 



picturized through the zoom lens and outputting as a video signal, a lens is 
driven to the zoom lens position coordinate ordered from an image connection 
means. Moreover, the position coordinate of a current zoom lens is supplied to 
an image connection means. 

[0008] A driving means (for example, punch Ruta) supplies the position 
coordinate of a current driving means to an image connection means while it can 
change the direction which drives an image pick-up means to a hand of cut, and 
image pick-up equipment picturizes and moves a driving means to the position 
coordinate ordered from an image connection means. An image connection 
means acquires the pan of current punch Ruta, and the location data of the 
direction of a tilt from a driving means, and calculates the include-angle 
coordinate of punch Ruta which makes a home position (for example, center 
position of the movable range of a driving means) the zero of the LAT and LONG 
while it acquires the zoom location of a current lens from an image pick-up 
means and calculates the scale factor. And a driving means is received when 
directions of panorama image creation occur from the interior of equipment (for 
example, periodic creation command), or the equipment exterior (for example, 
an operator's command). In order to acquire the image for creating a panorama 
image, a position coordinate is directed, an image is acquired after 
target-position attainment, compression of the image to the map to the virtual 



spherical surface of the acquired image, a horizontal, and a perpendicular 
direction, connection processing to the image captured previously, etc. are 
performed, and the connected image is outputted to a display means. And the 
image is connected one by one in such a procedure, and an image is saved 
while outputting the panorama image obtained eventually to a display means. In 
addition, in case the target position to a driving means is set up, an image 
connection means gives directions to a driving means so that the absolute 
location of a driving means may not be out of order under the effect by backlash, 
and it may advance into a target position from it being the same. 
[0009] 

[Embodiment of the Invention] Hereafter, some operation gestalten of this 
invention are explained with reference to a drawing. Drawing 11 shows the 
rough configuration of 1 operation gestalt of this invention. The computer 1 to 
which the monitor 2 and the mouse 8 are connected controls actuation of the 
punch Ruta camera 3 connected to the remote place. That is, the controller of 
image pick-up equipment is constituted by the computer 1. 
[0010] The punch Ruta camera 3 means what the punch Ruta section and the 
camera section consisted of in one. In this drawing 1 , the punch Ruta camera 3 
is installed in the actual scene in an environment as shown in 4 as an example. 
The screen (a photography screen is called hereafter) photoed with the punch 



Ruta camera 3 installed in the actual scene 4 of this environment is shown in 5. 
This photography screen is a screen currently photoed actually, if a zoom lens is 
operated to a looking-far side, a field angle will become small, and a field angle 
will become large if it is operated to a wide side. 

[0011] The image data which were supplied to the computer 1 and supplied to 
the computer 1 are decoded through a video cable etc., and the image of the 
photography screen 5 incorporated with the punch Ruta camera 3 is displayed 
on actuation field 6A of a monitor 2. In the monitor 2, it has panorama image 
display field 6B. Moreover, by panorama generation carbon button 6E If 
panorama screen generation directions are made, while driving punch Ruta and 
a zoom lens to a position and acquiring an image in each location An image is 
connected to the virtual spherical surface which made the movable shaft of a 
punch Ruta camera the zero for the acquired image of two or more sheets, and 
the flat-surface image which normalized to the LAT of the virtual spherical 
surface and the direction of LONG is displayed. Panorama actuation field 6B is 
made as [ operate / the punch Ruta camera 3 ] by directing one point of the 
arbitration in the image or panorama image with which the cursor 7 of an 
arrow-head form is drawn by the mouse 8 in the location, or field 6C of 
arbitration with a mouse 8. Furthermore, as a dotted line shows, punch Ruta 
limiter display 6D is displayed on panorama actuation field 6B. Punch Ruta liter 



display 6D shows the limitation of the movable range of the punch Ruta camera 
3. 

[0012] As shown in drawing 2 , on the screen of a monitor 2, actuation field 6A 
and panorama actuation field 6B are displayed. By operating a mouse 8, cursor 
can move and one point of the arbitration of panorama actuation field 6B or one 
point of the arbitration generated from field 6C of arbitration can be directed. 
Punch Ruta is made to drive and the image pick-up screen 5 moves so that one 
point of the directed arbitration may become in the center of actuation field 6A. 
That is, the result which you want to display beforehand is inputted and the 
photographic subject chosen according to the input is displayed to become in the 
center of actuation field 6A. 

[0013] Drawing 3 is the block diagram of all the systems that applied one gestalt 
of implementation of this invention. This system consists of the camera section 
11, the punch Ruta section 12, the TV monitor 13, a computer 1, a pointing 
device 14 of mouse 8 grade, and a monitor 2. Moreover, the punch Ruta camera 
3 mentioned above consists of the camera section 11 and the punch Ruta 
section 12, and the camera section 11 is installed on the punch Ruta section 12 
as an example. The camera section 11 consists of the lens block section 15, a 
zoom lens 16, the zoom section 17, the motor 18 for zoom lens 16, a solid state 
image pickup device 19, signal separation / automatic-gain-control circuit 20 



(SH/AGC), A/D converter 21, and a digital disposal circuit 22, and shows a video 
camera as a whole. 

[0014] The punch Ruta section 12 consists of the mode controller 23, the 
camera controller 24, the punch Ruta controller 25, a motor 26 for pans, a motor 
27 for tilts, and punch Ruta 28. A computer 1 consists of a control section 31 , the 
video capture section 29 which consists of a video capture board, and the 
storage section 30. 

[0015] Image formation of the image pick-up light which reaches from a 
photographic subject is carried out to a solid state image pickup device 19 
through the lens group of the lens block section 15, and drawing. There is CCD 
(Charge Coupled Device) as an example of a solid state image pickup device 19. 
After the image pick-up light by which image formation was carried out changes 
a visual field image into a video signal, it is supplied to signal separation / 
automatic-gain-control circuit 20. In signal separation / automatic-gain-control 
circuit 20, while sample hold of the image output signal is carried out, gain 
control is carried out so that it may have predetermined gain with the control 
signal of an auto iris (AE). The image output signal acquired is supplied to a 
digital disposal circuit 22 through A/D converter 21 by it. In a digital disposal 
circuit 22, the inputted signal is changed into each signals, such as brightness 
(Y), a color (C), and a video signal, and is supplied to the video capture section 



29 of a computer 1 through the TV monitor 13 as a video signal. 
[0016] Moreover, it is made possible to change the field angle picturized by 
driving a zoom lens 16 of the lens block section 15 of the camera section 11. 
This lens block section 15 rotates the motor 18 which consists of a stepping 
motor, and makes a zoom lens 16 drive with the actuation instruction of the 
camera controller 24 of the punch Ruta section 12. This camera controller 24 is 
also performing the interface with the mode controller 23 while being a controller 
which usually performs lens control (for example, a focus, a zoom, etc.) of the 
camera section 11, exposure control (for example, extracting gain, electronic 
shutter speed, etc.), white balance control, image quality control, etc. While 
outputting a control signal to Motor Driver as interface control relevant to control 
of a zoom lens 16 to the actuation instruction of the zoom lens 16 sent from the 
mode controller 23 so that it may drive in the location where the zoom lens 16 
was ordered, the positional information of the current zoom lens 16 always 
communicates for the mode controller 23. 

[0017] Moreover, the camera section 11 is installed on the punch Ruta section 
12 which are a pan and equipment with the degree of freedom of a biaxial hand 
of cut called a tilt. With the actuation instruction of the punch Ruta controller 25, 
the punch Ruta section 12 rotates the motor 26 for pans, and the motor 27 for 
tilts, and the universal head of punch Ruta 28 drives it respectively. There is a 



stepping motor as an example of these motors 26 and 27. while, as for this 
punch Ruta controller 25, a pan and the universal head of each tilt output a 
control signal to Motor Driver to the pan and the actuation instruction of the 
direction of each tilt which are sent from the mode controller 23 so that it may 
drive in the ordered location, the positional information of the pan of current 
punch Ruta 28 and the universal head of each tilt always communicates for the 
mode controller 23. 

[0018] The mode controller 23 controls the whole system according to the 
interface information from the internal state of the camera section 1 1 and the 
punch Ruta section 12, and the outside of the punch Ruta camera 3 to mention 
later. The mode controller 23 is connected with a computer 1 by RS-232C, and 
while distributing an instruction to the punch Ruta controller 25 and the camera 
controller 24 to the actuation instruction from a computer 1 so that the zoom lens 
16 of punch Ruta 28 and the lens block section 15 may be driven, the current 
positional information sent from the punch Ruta controller 25 and the camera 
controller 24 is transmitted to a computer 1. 

[0019] With one gestalt of this operation, in order to choose the image which the 
punch Ruta camera 3 projects, the computer 1 is used. And the commo data to 
the mode controller 23 is determined by processing the information acquired 
from the graphical display displayed on actuation field 6A on the screen of a 



monitor 2, and panorama actuation field 6B, and actuation of the directions 
location of a pointing device 14 (mouse 8), a click, etc. Moreover, in order to 
display the image of the camera section 11 on a monitor 2, the video capture 
section 29 is used. This video capture section 29 can be captured in the quality 
of arbitration with a capture signal to the graphics formats (for example, the still 
picture of a bit map format and a JPEG format, the animation of a JPEG format, 
etc.) of arbitration, and can carry out storage on the storage section 30 (for 
example, hard disk) of a computer 1 while it can display the video signal inputted 
from the camera section 1 1 on a monitor 2 in the quality of arbitration. 
[0020] Here, rough explanation for creating the panorama image in one gestalt 
of this operation is given using drawing 4 . First, let the surrounding environment 
where the punch Ruta camera 3 is set up be the spherical surface. This is called 
the virtual spherical surface. In this drawing 4 , the image of two ****** on the 
virtual spherical surface is connected, and the panorama image of one sheet is 
created. First, in order to create a panorama image, as shown in drawing 4 A, 
the punch Ruta camera 3 located at the core photos the image of two ****** on 
the virtual spherical surface. The punch Ruta camera 3 picturizes the flat surface 
which intersects perpendicularly with the optical axis of a lens. Drawing 4 D 
shows the condition that these two images were mapped by the flat surface 
which intersects perpendicularly with an optical axis, by picturizing the image of 



two ****** on the virtual spherical surface with the punch Ruta camera 3. When 
the image of two ****** is connected simply, there are duplication of a knot and a 
part which distortion produces. 

[0021] Duplication of this knot, and in order to lose distortion, as shown in 
drawing 4 B, the image of two ****** is mapped in the virtual spherical surface, 
respectively. Drawing 4 E shows the condition of having mapped two image 
pick-up images which are the flat surfaces which intersect perpendicularly with 
an optical axis in the virtual spherical surface. Thus, the flat surface which 
intersects perpendicularly with an optical axis, i.e., an image pick-up image, is 
mapped to the virtual spherical surface, the mapped image is connected, and 
connection processing of an image in which deletion of a duplication image and 
an unnecessary image is performed is performed. And by normalizing the image 
mapped by the virtual spherical surface at the LAT and LONG, as shown in 
drawing 4 C and drawing 4 D, a panorama image is generable. 
[0022] Next, law is explained while creating the panorama image by this 
invention. By this one approach, as shown in drawing 5 , the image of ten sheets 
is connected and it considers as the panorama image of one sheet. First, the 
image of ten sheets is photoed from the punch Ruta camera 3 (not shown) 
arranged at the core of the virtual spherical surface as shown in drawing 5 A. At 
this time, the punch Ruta camera 3 can acquire each image of 1-10 by setting 



the optical axis of the lens of the punch Ruta camera 3 by the location shown 
with a circle for every image field as shown in drawing. The image picturized with 
the punch Ruta camera 3 is an image on the flat surface which intersects 
perpendicularly with the optical axis of a lens, as shown in drawing 5 B. After 
being developed on the virtual spherical surface, respectively, the acquired 
image is normalized at the LAT and LONG, as shown in drawing 5 C. In the case 
of connection processing, as for the image often sheets developed on the virtual 
spherical surface, acquisition of an image is performed to a knot in a location 
where each other image overlaps so that there may be no omission. And after 
deletion of a duplication image and an unnecessary image is made, the image of 
ten sheets is connected, and a panorama image is generated as shown in 
drawing 5 D. 

[0023] Next, other approaches of creating the panorama image by this invention 
are explained with reference to drawing 6 . It is computed which pixel of the 
image acquired with the punch Ruta camera 3 is assigned, the pixel (s, t), i.e., 
each coordinate, of the panorama image which it normalizes at the LAT and 
LONG. When the pixel of the image acquired with the punch Ruta camera 3 is 
made equivalent to the pixel of a panorama image like the approach of drawing 
5 , it is for making the pixel of the image which the pixel of a panorama image 
without a corresponding pixel may produce, and was acquired to all the pixels of 



a panorama image correspond. Thus, a panorama image is realized by the pixel 
computed for every coordinate. As a procedure of the processing, the 
include-angle coordinate (alpha, beta) on the virtual spherical surface 
corresponding to the coordinate (s, t) ( drawing 6 A) of a panorama image 
( drawing 6 B) is first computed using a formula (1). 
[0024] 

(alpha, beta) = (a (s),b(t))(1) 

The detail of this formula (1) is explained in drawing 7 mentioned later. 

[0025] As the wide edge of image pick-up equipment is indicated to be the 

include-angle coordinate (theta, phi) of punch Ruta 28 which acquired this 

coordinate (s, t) and image to drawing 6 C using the image pick-up scale factor 

gamma made into 1 time, the coordinate data on the acquired image (xi, eta) is 

computed using a formula (2). 

[0026] 

(xi, eta) = (f (alpha, beta, theta, phi, gamma), g (alpha, beta, theta, phi, gamma)) 
(2) 

The detail of this formula (2) is explained in drawing 8 mentioned later. 
[0027] The connection image, i.e., a panorama image, is generated by matching 
each pixel of a panorama image, and the acquired image using the above 
formula. 



[0028] Here, how to change into the include-angle coordinate on the virtual 
spherical surface (alpha, beta) the coordinate (s, t) of the panorama image 
mentioned above is explained using drawing 7 . First, PragMin shown in drawing 
7 A is include-angle data of the left end when setting the home position of punch 
Ruta 28 to 0 (rag), and is PragMax. It is include-angle data of the right end when 
setting the home position of punch Ruta 28 to 0 (rag). Moreover, Ny2 It is the 
horizontal coordinate of panorama actuation field 6B, -Ny2 / 2 are coordinate 
data at the left end of panorama actuation field 6B, and Ny2 / 2 are coordinate 
data at the right end of panorama actuation field 6B. 

[0029] And in order to ask for the paninclude angle alpha from coordinate data s, 
it is set to (PragMax-alpha):(PragMax-PragMin)=:(Ny2 / 2-s) Ny2, and the 
paninclude angle alpha is alpha=PragMax from this. - (PragMax-PragMin) It is 
set to x / (Ny2 / 2-s) Ny2. 

[0030] Moreover, TragMin shown in drawing 7 B It is include-angle data of the 
upper bed when setting the home position of punch Ruta 28 to 0 (rag), and is 
TragMax. It is include-angle data of the soffit when setting the home position of 
punch Ruta 28 to 0 (rag). Moreover, Nz2 It is the coordinate of the perpendicular 
direction of panorama actuation field 6B, -Nz2 / 2 are coordinate data of the 
upper bed of panorama actuation field 6B, and Nz2 / 2 are coordinate data of the 
soffit of panorama actuation field 6B. 



[0031] And in order to ask for beta whenever [ tilt angle ] from coordinate data t, 
it is set to (TragMax-beta):(TragMax-TragMin)=:(Nz2 / 2-t) Nz2, and beta is 
beta=TragMax whenever [ tilt angle ] from this. - (TragMax-TragMin) It is set to x 
/ (Nz2 / 2-t) Nz2. 

[0032] With reference to drawing 8 , processing of flat-surface spherical-surface 
conversion is explained. As shown in drawing 8 A, the coordinate on the space 
of the point (xi, eta) on the camera image which turned to the home position 
(zero of the LAT and LONG) can be expressed as follows. 
[0033] 
[Equation 1] 

P = + I e i + k z n e A 
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[0034] At this time, it is k1 =tan(lambda/2gamma)/(Ny/2). 
k2 =tan(mu/2gamma)/(Nz/2) 



A next door, and (Ny, Nz) are the actuation range (the direction of y, the direction 
of z) of a pointing device 14 (mouse 8), (lambda, mu) are a horizontal in a wide 
edge, and a vertical field angle, and gamma is a current zoom relative 
magnification (scale-factor information) which makes a wide edge 1 time. 
[0035] Moreover, as shown in drawing 8 B, generally it is known that the 
revolution matrix of a three dimension is shown like a degree type. 



[0036] 
[Equation 2] 
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[0037] Since there is only a home position to include-angle information (theta, 
phi) in the direction where (alpha, beta) have one same point, the following 
relation consists of 1 on a pan and the camera image which carried out the tilt (xi, 
eta), and a home position. 



[0038] 

Rz Ry (theta) p=IRz (phi) Ry (alpha) ex (beta) - if this is solved about p -- [0039] 



[Equation 3] 

P = / R y (-4>) R x (a - 6) R y (p) e, 

cos (a - 0) coscf>cosp + sin<t>sinp 

= / sin (or - 0) cosp 

-cos (a - 0) sin<t>cosg + cos$sin(3 



[0040] It is here and is [0041]. 



[Equation 4] 



P = / 



a 
b 

c 



(4) 



[0042] If it sets, xi and eta will be called for as follows. 

[0043] 1=1 / axi=-lb/k1 - it projects on an image pick-up coordinate from the 
coordinate data which is in the include angle of (alpha, beta) from a home 
position from the formula more than =-b/k1a eta=lc/k 2 =c/k2a - having (xi, eta) 



it can ask. 



[0044] xi=(-sin(alpha-theta) cosbeta)/() [ k1 ] (cos(alpha-theta) cosphi 



cosbeta+sinphi sinbeta) eta=(-cos(alpha-theta) sinphi cosbeta+cosphi 



sinbeta)/(k2) - thus, (cos(alpha-theta) cosphi cosbeta+sinphi sinbeta) By asking 
for the coordinate data (xi, eta) on the image acquired from the include-angle 
coordinate (alpha, beta) on the virtual spherical surface corresponding to the 
coordinate (s, t) of a panorama image with the punch Ruta camera 3, a 
panorama image is generable. 

[0045] conversely, it projects on an image pick-up coordinate from the following 
formulas - having (xi, eta) - it can also ask for the coordinate data in the include 
angle of (alpha, beta). 

[0046] First, since it is l=|p|, it is a=1/root (1+k1 2 xi2+k2 2 eta2). 
b=-k1 xi/root (1+k1 2 xi2+k2 2 eta2) 
c=k2 eta/root (1+k1 2 xi2+k2 2 eta2) 

It becomes. However, root ( ) processes a square root to the count result in ( ). 
[0047] Moreover, it is set to a=cos(alpha-theta) cosphi cosbeta+sin phi 
sinbetab=sin (alpha-theta) cosbetac=-cos(alpha-theta) sinphi cosbeta+cosphi 
sinbeta from a formula (3). 

[0048] It is a sin phi+c sin theta=-1 (sinphi/root(1+k1 2 xi2+k2 2 eta2)+sinthetak2 

eta/root (1+k1 2 xi2+k2 2 eta2)) from the above formula, sinbetatan (alpha-theta) 

Since it is =b/(a cosphi-c sintheta), it is beta=sin. 

Alpha=tan It is set to -1(-k1 xi/(cos phi-k 2 eta sintheta))+theta. 

[0049] Therefore, = (alpha, beta) (f (xi, eta, theta, phi, gamma), g (xi, eta, theta, 



phi, gamma)) (4) 

******** — things are made. 

[0050] moreover - if some errors are permitted - being as follows (alpha, beta) 
— it can ask. 

[0051] Alpha=theta + (lambda/gamma) x (xi/Ny) 
Beta=phi + (micro/gamma) x (eta/Nz) 

That is, formula (4) (alpha, beta) = (f (xi, theta, gamma), g (eta, phi, gamma)) (5) 
It can become and simplify. 

[0052] The processing which computes the include-angle information (alpha, 
beta) on punch Ruta 28 shown in the formula (4) and formula (5) which were 
mentioned above from the position coordinate (xi, eta) of actuation field 6A is 
explained using drawing 9 . First, an example of an approach which specifies 
one point of the arbitration in actuation field 6A directly is explained. As shown in 
drawing 9 A, it considers as the relative coordinate which set the core of 
actuation field 6A to (0, 0), and the position coordinate (xi, eta) of the mouse 8 on 
the actuation field 6A is acquired. 

[0053] Next, an example of other approaches which specifies one point of the 
arbitration generated from the field of the arbitration in actuation field 6A is 
explained. First, it is a main coordinate of the square which the end-of-region 
point (m2, n2) of arbitration is specified, and is generated from these two points 



after the starting point (ml, n1) of the field of arbitration is specified, as shown in 
drawing 9 A. (xi, eta) = (ml, n1) (+ (m2, n2))/2 (6) 
It is alike and one point (xi, eta) of arbitration is acquired more. 
[0054] This drawing 9 A is the coordinate of the mouse 8 (pointing device 14) on 
actuation field 6A, and sets the movable range of the mouse 8 in actuation field 
6A (the direction of y, the direction of z) to (Ny1 and Nz1). The include-angle 
coordinate (alpha, beta) of punch Ruta 28 is searched for from a formula (4) or a 
formula (5) using the position coordinate (xi, eta) of one point (mouse 8) of this 
arbitration, the include-angle information (theta, phi) which punch Ruta 28 has 
turned to, and the scale-factor information (gamma) made into the current zoom 
relative magnification which makes the wide edge of a zoom lens 16 1 time. 
[0055] The include-angle coordinate (alpha, beta) shown in drawing 9 B is for 
projecting on the core of an image pick-up screen the location specified with the 
pointing device, when the home position of punch Ruta 28 is made into the zero 
oftheLATand LONG. 

[0056] In this drawing 9 , the absolute coordinate of the screen of a monitor 2 is 
sufficient as the coordinate searched for, and the relative coordinate which set 
the core of actuation field 6A to (0, 0) is sufficient as it. This coordinate shows 
the coordinate of the direction of a pan by xi, ml, m2, theta, and alpha, and 
shows the coordinate of the direction of a tilt by eta, n1, n2, phi, and beta. 



[0057] Thus, the include-angle information on current punch Ruta 28 obtained 
with received data when a mouse 8 was in actuation field 6A (theta, phi), The 
photographic subject on specified actuation field 6A computes the include-angle 
information on punch Ruta 28 (alpha, beta) that it comes to the core of actuation 
field 6A, using a formula (4) or a formula (5) using the scale-factor information 
(gamma) on a zoom, and the positional information (xi, eta) of the mouse 8 of 
the field specified with the mouse 8. In this way, according to drawing 11 , the 
include-angle coordinate (alpha, beta) of obtained punch Ruta 28 is changed 
into the internal positional information (PNew, TNew) of punch Ruta 28, and is 
stored in a transmission buffer with the absolute location actuation instruction of 
punch Ruta 28. Moreover, it is a data Request-to-Send flag (FlagSo) 
simultaneously so that it may mention later. It sets and data are transmitted in a 
timer event. 

[0058] The processing which changes into an include-angle coordinate (alpha, 
beta) the position coordinate (xi, eta) of the mouse 8 of panorama actuation field 
6B with which a panorama image is displayed is explained using drawing 10 . 
The method of specifying one point of the arbitration in panorama actuation field 
6B directly can acquire the position coordinate (xi, eta) of a mouse 8, as it is the 
approach of specifying directly, and the same approach and one point of the 
arbitration in actuation field 6A mentioned above is shown in drawing 10 A. 



[0059] Next, an example of other approaches which specifies one point of the 
arbitration generated from the field of the arbitration in panorama actuation field 
6B is explained. First, as shown in drawing 10 A, after the starting point (mt, n1) 
of the field of arbitration is specified, the end-of-region point (m2, n2) of 
arbitration is specified, and one point (xi, eta) of arbitration is acquired by the 
formula (6). 

[0060] This drawing 10 A is the coordinate of the mouse 8 (pointing device 14) 
on panorama actuation field 6B, and sets the movable range of the mouse 8 in 
panorama actuation field 6B (the direction of y, the direction of z) to (Ny2 and 
Nz2). This movable range is restricted in panorama actuation field 6B by punch 
Ruta limiter display 6D shown by the dotted line. This punch Ruta limiter display 
6D shows the movable range of the optical axis of the lens of the punch Ruta 
camera 3. That is, it cannot direct more than punch Ruta limiter display 6D. The 
position coordinate of one point of this arbitration (xi, eta), and the include-angle 
information which punch Ruta 28 has turned to (theta, phi), The include-angle 
coordinate (alpha, beta) of the position coordinate (x y) of panorama actuation 
field 6B, field angle information (s, t), and punch Ruta 28 is searched for from a 
formula (7), a formula (8), and a formula (9) using the scale-factor information 
(gamma) made into the current zoom relative magnification which makes the 
wide edge of a zoom lens 16 1 time. 



[0061] 

(x y) = (fO (theta) and gO) (phi) (7) 

(s, t) = (f1 (gamma) and g1) (gamma) (8) 

(alpha, beta) = (f (xi), g (eta)) (9) 

The position coordinate (x y) shown in drawing 10 B is the sense of present 
punch Ruta 28, when the home position of punch Ruta 28 is made into the zero 
of the LAT and LONG, and field angle information (s, t) is a field angle currently 
displayed on present actuation field 6A. This drawing 10 B displays a zoom on 
panorama actuation field 6B, and the condition of punch Ruta. 
[0062] The include-angle coordinate (alpha, beta) shown in drawing 10 C is for 
projecting on the core of an image pick-up screen the location specified with the 
pointing device, when the home position of punch Ruta 28 is made into the zero 
of the LAT and LONG. (PragMax, TragMax) - (PragMin, TragMin) is the range of 
punch Ruta which can be driven, i.e., the range shown by punch Ruta limiter 
display 6D. This drawing 10 C expresses the actuation desired value on the 
punch Ruta movable range. 

[0063] In this drawing 10 , the absolute coordinate of the screen of a monitor 2 is 
sufficient as the coordinate searched for, and the relative coordinate which set 
the core of panorama actuation field 6B to (0, 0) is sufficient as it. This 
coordinate shows the coordinate of the direction of a pan by xi, ml , m2, x, s, and 



alpha, and shows the coordinate of the direction of a tilt by eta, n1, n2, y, t, and 
beta. 

[0064] Thus, when a mouse 8 is in panorama actuation field 6B, the 
photographic subject on specified actuation field 6A computes the include-angle 
information on punch Ruta 28 (alpha, beta) that it comes to the core of actuation 
field 6A, using a formula (9) using the positional information (xi, eta) of the 
mouse 8 of the field specified with the mouse 8. In this way, according to 
drawing 11 , the include-angle coordinate (alpha, beta) of obtained punch Ruta 
28 is changed into the internal positional information (PNew, TNew) of punch 
Ruta 28, and is stored in a transmission buffer with the absolute location 
actuation instruction of punch Ruta 28. Moreover, a data Request-to-Send flag 
(FlagSo) is set simultaneously, and data are transmitted in a timer event so that 
it may mention later. 

[0065] Here, how to change an include-angle coordinate (alpha, beta) into the 
positional information (PNew and TNew) of the punch Ruta 28 interior is 
explained to the approach list which changes into include-angle information 
(theta, phi) the positional information (p, t) of the punch Ruta 28 interior 
mentioned above using drawing 11 . First, PragMin shown in drawing 11 A It is 
include-angle data of the left end when setting the home position of punch Ruta 
28 to 0 (rag), and is PragMax. It is include-angle data of the right end when 



setting the home position of punch Ruta 28 to 0 (rag). Moreover, PdatMin It is 
internal count data at the left end of the punch Ruta controller 25, and is 
PdatMax. It is internal count data at the right end of the punch Ruta controller 25. 
[0066] And in order to ask for the paninclude angle theta from the pandata p, it 
becomes 

(PragMax-theta):(PragMax-PragMin)=(PdatMax-p):(PdatMax-PdatMin), and the 
paninclude angle theta is theta=PragMax from this. It becomes 
-(PragMax-PragMin)x(PdatMax-p)/(PdatMax-PdatMin). 

[0067] Moreover, the pandata p serve as p=PdatMax-(PragMax-theta) 
x(PdatMax-PdatMin)/(PragMax-PragMin) from this. 

[0068] Moreover, in order to ask for the pandata PNew from the paninclude 
angle alpha, it becomes 

(PragMax-alpha):(PragMax-PragMin)=(PdatMax-p-new):(PdatMax-PdatMin) 
and the pandata PNew serve as PNew=PragMax-(PragMax-alpha) 
x(PdatMax-PdatMin)/(PragMax-PragMin) from this. 

[0069] Moreover, TragMin shown in drawing 11 B It is include-angle data of the 
upper bed when setting the home position of punch Ruta 28 to 0 (rag), and is 
TragMax. It is include-angle data of the soffit when setting the home position of 
punch Ruta 28 to 0 (rag). Moreover, TdatMin It is internal count data of the upper 
bed of the punch Ruta controller 25, and is TdatMax. It is internal count data of 



the soffit of the punch Ruta controller 25. 

[0070] And in order to ask for phi whenever [ tilt angle ] from the tilt data t, it 
becomes (TragMax-phi):(TragMax-TragMin)=(TdatMax-t):(TdatMax-TdatMin), 
and phi is phi=TragMax whenever [ tilt angle ] from this. It becomes 
-(TragMax-TragMin)x(TdatMax-t)/(TdatMax-TdatMin). 

[0071] Moreover, the tilt data t serve as t=TdatMax-(TragMax-phi) 
x(TdatMax-TdatMin)/(TragMax-TragMin) from this. 

[0072] Moreover, it is (TragMax-beta):(TragMax-TragMin)=(TdatMax-TNew) in 
order to ask for the tilt data TNew from whenever [ tilt angle / beta ]. : 
(TdatMax-TdatMin) Becoming, the tilt data TNew are 
TNew=TragMax-(TragMax-beta) x(TdatMax-TdatMin)/(TragMax-TragMin) from 
this. 

It becomes. 

[0073] Next, how to change the include-angle information on punch Ruta 28 
(theta, phi) into the position coordinate in panorama actuation field 6B (x y) is 
explained to the approach list which changes into the include-angle coordinate 
(alpha, beta) of punch Ruta 28 the position coordinate in panorama actuation 
field 6B mentioned above (xi, eta) using drawing 12 . First, PragMin shown in 
drawing 12 A It is include-angle data of the left end when setting the home 
position of punch Ruta 28 to 0 (rag), and is PragMax. It is include-angle data of 



the right end when setting the home position of punch Ruta 28 to 0 (rag). 
Moreover, Ny2 It is the horizontal coordinate of panorama actuation field 6B, 
-Ny2 / 2 are coordinate data at the left end of panorama actuation field 6B, and 
Ny2 / 2 are coordinate data at the right end of panorama actuation field 6B. 
[0074] And in order to ask for the paninclude angle alpha from coordinate data xi, 
it is set to (PragMax-alpha):(PragMax-PragMin)=:(Ny2 / 2-xi) Ny2, and the 
paninclude angle alpha is alpha=PragMax from this. - (PragMax-PragMin) It is 
set to x / (Ny2 / 2-xi) Ny2. 

[0075] Moreover, in order to ask for coordinate data x from the paninclude angle 
theta, it is set to (PragMax-theta):(PragMax-PragMin)=:(Ny2 / 2-x) Ny2, and 
coordinate data x becomes x=Ny2 / 2-(PragMax-theta) 
xNy2/(PragMax-PragMin) from this. 

[0076] Moreover, TragMin shown in drawing 12 B It is include-angle data of the 
upper bed when setting the home position of punch Ruta 28 to 0 (rag), and is 
TragMax. It is include-angle data of the soffit when setting the home position of 
punch Ruta 28 to 0 (rag). Moreover, Nz2 It is the coordinate of the perpendicular 
direction of panorama actuation field 6B, -Nz2 / 2 are coordinate data of the 
upper bed of panorama actuation field 6B, and Nz2 / 2 are coordinate data of the 
soffit of panorama actuation field 6B. 

[0077] And in order to ask for beta whenever [ tilt angle ] from coordinate data 



eta, it is set to (TragMax-beta):(TragMax-TragMin)=:(Nz2 / 2-eta) Nz2, and beta 
is beta=TragMax whenever [ tilt angle ] from this. - (TragMax-TragMin) It is set to 
x / (Nz2 / 2-eta) Nz2. 

[0078] Moreover, in order to ask for coordinate data y from whenever [ tilt angle / 
phi ], it is set to (TragMax-phi):(TragMax-TragMin)=:(Nz2 / 2-y) Nz2, and 
coordinate data y becomes y=Nz2 / 2-(TragMax-theta) xNz2/(TragMax-TragMin) 
from this. 

[0079] How to change into the field angle information on picture frame 6C in 
panorama actuation field 6B (s, t) from the field angle information (psi, omega) 
which punch Ruta 28 is starting is explained using drawing 13 . First, the field 
angle information on present punch Ruta 28 (psi, omega) is shown in drawing 13 
A. This field angle information (psi, omega) is =(psi, omega) 1/gammax (psiO, 
omegaO). 

It asks "Be alike." At this time, (psiO, omegaO) show the horizontal angles of view 
and the vertical field angle in a wide edge, and gamma shows the lens scale 
factor when making a wide edge into 1 time. 

[0080] It is PragMin as shown in drawing 13 B. It is include-angle data of the left 
end when setting the home position of punch Ruta 28 to 0 (rag), and is PragMax. 
It is include-angle data of the right end when setting the home position of punch 
Ruta 28 to 0 (rag). Moreover, Ny2 It is the horizontal coordinate of panorama 



actuation field 6B, -Ny2 / 2 are coordinate data at the left end of panorama 
actuation field 6B, and Ny2 / 2 are coordinate data at the right end of panorama 
actuation field 6B. 

[0081] And in order to search for horizontal angles of view s from horizontal 
angles of view psi, it becomes psi:(PragMax-PragMin) =s:Ny2 and horizontal 
angles of view s become s=psixNy 2/(PragMax-PragMin) from this. 
[0082] Moreover, TragMin shown in drawing 13 C It is include-angle data of the 
soffit when setting the home position of punch Ruta 28 to 0 (rag), and is 
TragMax. It is include-angle data of the upper bed when setting the home 
position of punch Ruta 28 to 0 (rag). Moreover, Nz2 It is the coordinate of the 
perpendicular direction of panorama actuation field 6B, -Nz2 / 2 are coordinate 
data of the soffit of panorama actuation field 6B, and Nz2 / 2 are coordinate data 
of the upper bed of panorama actuation field 6B. 

[0083] And in order to ask for the vertical field angle t from the vertical field angle 

omega, it becomes omega:(TragMax-TragMin) =t:Nz2 and the vertical field 

angle t becomes t=omegaxNz 2/(TragMax-TragMin) from this. 

[0084] From these, the field angle information (s, t) shown in drawing 13 D is 

displayed as picture frame 6C in panorama actuation field 6B. 

[0085] Next, how to change into scale-factor information (gamma) the positional 

information (z) of the zoom lens 16 mentioned above is explained using drawing 



14 . This drawing 14 expresses lens scale-factor information to an axis of 
ordinate, and expresses the internal information of a zoom lens with an axis of 
abscissa. Positional information (z) of the acquired zoom lens 16 is compared 
and made into the conversion graph shown in drawing 14 , and is changed into 
scale-factor information (gamma) on a computer 1. Positional information (z) is 
changed into scale-factor information (gamma) by a ROM table or the formula as 
an example. 

[0086] Next, an example of the control algorithm in a computer 1 is explained 
using drawing 15 . First, at step S1 , a start of a program performs initialization to 
which actuation field 6A, panorama actuation field 6B, cursor 7, and the punch 
Ruta limiter further shown in punch Ruta limiter display 6D are set on a monitor 2, 
as shown in drawing 2 . A fixed value is sufficient as the range of a punch Ruta 
limiter, and you may enable it to change it freely to change the range. And at 
step S2, in order that a computer 1 and the mode controller 23 may 
communicate with a predetermined period, a timer is set up. If these initialization 
actions are completed, control will move to the event waiting state in which step 
S3 carries out various generating, and control will move from step S3 to it 
corresponding to the generated event. The event to generate has the timer event 
(step S4) and panorama creation request event (step S5) which were set up 
previously. 



[0087] The detail of the algorithm of a panorama creation request event is 
explained using the flow chart of drawing 16 . Generating of a panorama 
creation request event sets a panorama creation demand (FlagPa) at step S8 
(True). 

[0088] The detail of the algorithm of a timer event is explained using the flow 
chart of drawing 17 . The timer event of this example is an event generated in 
order to perform the communication link with a computer 1 and the mode 
controller 23 periodically. This timer event is generated at intervals of 50msec as 
an example. Generating of a timer event judges whether setting out of a 
communication link port is completed at step S11. setting out of a 
communication link port is completed (settled) - ** - if judged, control will move 
to step S12 and setting out of a communication link port will not be completed - 
** (sheep) - if judged, control will move to step S18. Here, control moves only 
from the first time which setting out of a communication link port has not 
completed to step S18, and establishment processing of a communication link 
port is performed. Specifically in step S18, establishment of the RS-232C port on 
a computer 1 is performed. 

[0089] In the timer event after it, transmitting processing of the commo data for 
transmitting processing of the data which have accumulated in transmission 
buffers, such as a check of received data, analysis processing, and an actuation 



instruction of punch Ruta 28, or punch Ruta 28, and the condition acknowledge 
request of a zoom lens 16 is performed. With this algorithm, by step S12, control 
moves from step S11 to step S12, and when the existence of the data of a 
receive buffer is checked and received data exist, control moves to step S13, 
and when received data do not exist, control moves to step S14. At step S13, the 
received data which exist in a receive buffer are analyzed, and the positional 
information (p, t) of punch Ruta 28 required of the mode controller 23 and the 
positional information (z) of a zoom lens 16 are acquired. It is changed into the 
include-angle information on punch Ruta 28 (theta, phi), and the scale-factor 
information (gamma) on a zoom lens 16 according to the method of drawing 11 
which these data mentioned above, and drawing 14 . 

[0090] Next, the existence of the Request to Send of data is checked at step S14. 
When the Request to Send of data exists (FlagSo==True), control moves to step 
S19, and after transmitting processing of the data stored in the transmission 
buffer is made, a Request-to-Send flag (FlagSo) is reset at step S19 (False). As 
an example of the data which accumulated in this transmission buffer, there are 
data of an actuation instruction of punch Ruta 28 set up with the mouse 8 etc. 
And when there is no Request to Send (FlagSo==False), control moves to step 
S15. At step S15, it is judged [ the internal counter (ReqCnt) of a Request to 
Send ] for 0 whether it is no, when the internal counter of a Request to Send is 0 



(ReqCnt=0), control moves to step S16, and when the internal counter of a 
Request to Send is not 0 (ReqCnt!=0), control moves to step S20. 
[0091] At step S16, the location demand instruction of punch Ruta 28 and a 
zoom lens 16 is transmitted to the mode controller 23. And at step S17, the 
increment of the internal counter (ReqCnt) of a Request to Send is carried out. 
[0092] At step S20, when it is judged whether a panorama creation demand 
exists and a panorama creation demand exists (FlagPa==True), control moves 
to step S21, and when a panorama creation demand does not exist 
(FlagPa==False), control moves to step S22. Panorama creation processing 
mentioned later is performed at step S21. And at step S22, the internal counter 
(ReqCnt) of a Request to Send is set to 0. 

[0093] Next, the algorithm of the panorama creation processing mentioned 
above is explained using the flow chart of drawing 18 . Panorama creation is set 
up by the panorama creation request event (FlagPa=True). If this panorama 
creation request event occurs beforehand, as mentioned above, panorama 
creation processing (step S21) will be performed at the time of a timer event. In 
this flow chart, the procedure of panorama creation processing is performed 
according to the panorama counter (PanoCnt). 

[0094] First, initiation of this processing performs processing which sets the field 
angle of image pick-up equipment as the maximum wide angle. That is, in step 



S31, when it is judged whether a panorama counter (PanoCnt) is 1 and a 
panorama counter is judged to be 1 (PanoCnt ==1), control moves to step S43, 
and when it is judged that a panorama counter is not 1 (PanoCnt !=1), control 
moves to step S32. At step S43, while transmitting the actuation instruction 
which moves a zoom to a wide edge, a panorama counter is incremented. This 
is for performing a wide range panorama image by small image acquisition time. 
[0095] At step S32, when it is judged whether a panorama counter is 2 and a 
panorama counter is judged to be 2 (PanoCnt ==2), control moves to step S44, 
and when it is judged that a panorama counter is not 2 (PanoCnt !=2), control 
moves to step S33. At step S44, while transmitting the actuation instruction 
which moves punch Ruta 28 to the position (POS (PanoCnt-2)) photoed first, a 
panorama counter is incremented. That is, processing which moves to the first 
location (location shown by 1 of drawing 5 ) which acquires the field angle of 
punch Ruta 28 shown by POS (PanoCnt-2) is performed. 

[0096] Moreover, the capture of an image is performed, when it checks that 
image pick-up equipment had moved to the set-up zoom and a punch Ruta 
location when a panorama counter (PanoCnt) was after three and arrives at the 
specified location. At step S33, when the check of the positional information (z) 
of a zoom lens 16 is performed and the zoom location suits, control moves to 
step S34, and when the zoom location does not suit, this flow chart is ended. 



Furthermore, at step S34, when the check of the positional information (p, t) of 
punch Ruta 28 is performed and the punch Ruta location suits (==POS 
(PanoCnt-3)), control moves to step S35, and when the punch Ruta location 
does not suit (!=POS (PanoCnt-3)), this flow chart is ended. And the image of a 
current punch Ruta location is acquired at step S35. The acquired image is 
memorized by memory as an image file of a bit map. 

[0097] At step S36, the actuation instruction which moves punch Ruta 28 to the 
position (POS (PanoCnt-2)) photoed next is transmitted. At step S37, as 
mentioned above, data conversion of the acquired image is carried out to the 
virtual spherical surface, and connection processing of an image in which 
deletion of a duplication image and an unnecessary image is performed is 
performed. Moreover, when there is no throughput of a computer 1, at this step 
S37, compression is also performed to horizontal and a perpendicular direction. 
At step S38, the connected image is normalized and displayed by the LAT and 
LONG. And the increment of the panorama counter (PanoCnt) is carried out at 
step S39. 

[0098] That is, deletion of the map processing to the virtual spherical surface of 
the image which performed processing which moves punch Ruta 28 to the 
image acquisition location of a degree, then carried out the capture previously, a 
horizontal, the compression processing to a perpendicular direction, a 



duplication image, and an unnecessary image etc. is performed, and the 
progress situation of panorama image creation is displayed. It carries out until a 
panorama image completes the above actuation. 

[0099] When it is judged whether the panorama image was completed 
thoroughly and it is judged that it completed, control moves to step S41, and at 
step S40, when it is judged that it is incomplete, it ends, and this flow chart is 
repeated until a panorama image completes above-mentioned control again. At 
step S41, since the panorama image was completed, a panorama creation 
demand (FlagPa) is reset (False). And the completed panorama image is saved 
at step S42. 

[0100] If the image of 10 shown in drawing 5 which can gain the image of 1 
shown in drawing 5 in this flow chart when a panorama counter (PanoCnt) is 3, 
can gain the image of 2 shown in drawing 5 when a panorama counter is 4, and 
is gained when a panorama counter is 12 is gained, all the images for generating 
a panorama image will be gained, and a panorama image will be completed. 
[0101] In addition, to the target position where it is ordered here, the driving gear 
operated so that it might always advance from homotopic, and it has amended 
the gap of the connection image by the play (for example, backlash of a gear) of 
the device of the drive system which may be generated in the case of image 
connection. 



[0102] A panorama image as shown in drawing 21 C from the image of ten 
sheets as shown in drawing 21 A according to 1 operation gestalt of this 
invention mentioned above can be obtained. 

[0103] With 1 operation gestalt mentioned above, it is made as [ perform / all 
control / using one computer ]. To be shown in drawing 19 , other operation 
gestalten of this invention share a role between the computer for servers, and 
the computer for clients, and control a punch Ruta camera also by the gestalt 
which has a limit in channel capacity like a network circuit. The computer 1 to 
which the monitor 2 and the mouse 8 are connected controls actuation of the 
punch Ruta camera 3 installed in the remote place through the transmission line 
and the server 9. That is, an image pick-up equipment controller is constituted by 
the computer 1. As a transmission line, the existing various things, such as a 
communication line (wireless, cable) and a network, are usable. A computer 1 
has the relation of a client to a server 9, and connection of two or more 
computers 1 is possible to a server 9. 

[0104] The punch Ruta camera 3 and a server 9 are installed in the actual scene 
in an environment as shown in 4. The screen (a photography screen is called 
hereafter) photoed with the punch Ruta camera 3 installed in the actual scene 4 
of this environment is shown in 5. This photography screen 5 is a screen 
currently photoed actually, if a zoom lens is operated to a looking-far side, a field 



angle will become small, and a field angle will become large if it is operated to a 
wide side. 

[0105] The image of the photography screen 5 incorporated with the punch Ruta 
camera 3 is changed into image data by going via a server 9. This image data is 
supplied to a computer 1 through a transmission line. The image data supplied 
to the computer 1 are displayed on a monitor 2. In a monitor 2, the supplied 
photography screen 5 is displayed on actuation field 6A on the screen of a 
monitor 2. 

[0106] Moreover, in the monitor 2, it has panorama actuation field 6B and 
panorama generation carbon button 6E. Creation directions of a panorama 
image are sent to a server 9 by panorama generation carbon button 6E, in a 
server 9, while driving punch Ruta and a zoom to a position and acquiring an 
image in each location, an image is connected to the virtual spherical surface 
which made the movable shaft of punch Ruta the zero for the acquired image of 
two or more sheets, and the flat-surface image which normalized to the LAT of 
the virtual spherical surface and the direction of LONG is created. The created 
panorama image is displayed on panorama actuation field 6B of a monitor 2 by 
computer 1 through a transmission line from a server 9. 

[0107] Moreover, the cursor 7 of an arrow-head form is drawn by panorama 
actuation field 6B like 1 operation gestalt mentioned above in the location of a 



mouse 8. The punch Ruta camera 3 is operated by directing one point or field 6C 
of the arbitration in an image or a panorama image with a mouse 8. Furthermore, 
as a dotted line shows, punch Ruta limiter display 6D is displayed on panorama 
actuation field 6B. Punch Ruta liter display 6D shows the limitation of the 
movable range of the punch Ruta camera 3. 

[0108] And by operating a mouse 8, cursor can be moved and can direct one 
point of the arbitration of panorama actuation field 6B, or one point of the 
arbitration generated from field 6C of arbitration. Punch Ruta is made to drive 
through a server 9 and a transmission line, and the image pick-up screen 5 
moves so that one point of the directed arbitration may become in the center of 
actuation field 6A. That is, it is displayed that the selected photographic subject 
becomes in the center of panorama actuation field 6B through a server 9 and a 
transmission line. 

[0109] Drawing 20 is the block diagram of all the systems of other operation 
gestalten of implementation of this invention. However, since the configuration of 
the camera section 11 and the punch Ruta section 12 and the function are the 
same as that of 1 operation gestalt ( drawing 3 ) mentioned above, the detailed 
configuration is omitted in drawing 20 . A server 9 consists of a control section 
131, the video capture section 129 which consists of a video capture board, and 
the storage section 30. It connects in the transmission line 132 and the network, 



and a computer 1 consists of control-section 31 grades like 1 operation gestalt 
shown in drawing 3 . In addition, also about the detailed algorithm in each 
computer, since said 1 operation gestalt and content overlap, the explanation is 
omitted. 

[0110] Like 1 operation gestalt, signal processing of the image pick-up light 
which reaches from a photographic subject is carried out in the camera section 
11, it is changed into each signals, such as brightness (Y), a color (C), and a 
video signal, and is supplied to the TV monitor 13 and the video capture section 
129 of a server 9 as a video signal. Moreover, as 1 operation gestalt, a punch 
Ruta camera has the mode controller 23, the camera controller 24, and the 
punch Ruta controller 25, and is controlling the camera section 1 1 and the punch 
Ruta section 28. The mode controller 23 controls the whole system like 1 
operation gestalt according to the internal state of the camera section 1 1 and the 
punch Ruta section 12, and the instruction from the outside. 
[0111] The mode controller 23 is connected with a server 9 according to a 
channel (specifically, RS232C is used), and the received instruction is 
distributed to the punch Ruta controller 25 and the camera controller 24 so that 
the zoom lens 16 of punch Ruta 28 and the lens block section 15 may be driven 
to the instruction sent by server 9 course from the instruction directly sent from a 
server 9, or a computer 1. Moreover, since the mode controller 23 sends out the 



internal state of a punch Ruta camera outside via a server 9, it always acquires 
information from the punch Ruta controller 25 and the camera controller 24. 
[0112] A server 9 acquires periodically the internal states (for example, punch 
Ruta, current positional information of a zoom lens, etc.) of a punch Ruta camera 
from the mode controller 23 of the punch Ruta section 12. Moreover, since the 
image of the camera section 1 1 is sent out to a transmission line 132, the video 
capture section 129 is used and the image information inputted from the camera 
section 11 has been changed into the digital image data (specifically the still 
picture of a JPEG format or the still picture of a bit map format) which is easy to 
send out to a transmission line 132 in the quality of arbitration. Moreover, 
storage of the digital image of an isomorphism type can be carried out on the 
storage section 130 (for example, hard disk). 

[0113] If a connection request is made from a computer 1 to a server 9, a server 
9 sends out the GUI (graphical interface) panel information for displaying on the 
monitor 2 connected to the computer 1. As panel information, there is a program 
of operation in the computer 1 when mouse actuation is carried out on 
arrangement of a panel and a panel etc., and, specifically, the program of HTML 
and JAVA is used. Moreover, the condition of the image data which the punch 
Ruta camera picturized periodically, and a punch Ruta camera etc. is sent out to 
a computer 1 through a transmission line 132. 



[01 14] With other operation gestalten, the Internet is used for a transmission line 
132 and picking is made in the transmission-line 132 top using HTTP pro 
TORUKO. Moreover, the GUI panel information sent from a server 9, image 
information, a punch Ruta camera condition, etc. are expressed to a monitor 2 
as a computer 1 using the browser for the Internet. Actuation field 6A and 
panorama actuation field 6B, the manual operation button of a zoom, the cursor 
of a pointing device 14 (mouse 8), etc. are displayed on the GUI panel displayed 
on the screen of a monitor 2. And a decoding indication of the image data sent 
from a server is given, and an image is rewritten by actuation field 6A with 
renewal of image data. Moreover, the panorama image data sent from a server 9 
is decoded, a panorama image is displayed on panorama actuation field 6B, and 
the operating range of a punch Ruta camera and the location of current punch 
Ruta, the field angle of a zoom, etc. are further expressed to it as the same 
technique as 1 operation gestalt. And a program of operation when the GUI 
panel by which the computer 1 has been sent from the server 9 is operated is 
executed. 

[0115] The above-mentioned program of other operation gestalten of operation 
generates an actuation instruction to a punch Ruta camera, and the instruction 
of operation on a server by click actuation of a mouse. When a mouse is clicked, 
on actuation field 6A like 1 operation gestalt Field angle information, current 



punch Ruta positional information, and the positional information of the mouse 
when being clicked to origin An instruction (absolutely a location actuation 
instruction or a relative-position actuation instruction) is sent to a server 9 so that 
the location by which the mouse click was carried out on the image displayed on 
actuation field 6A may come to the core of actuation field 6A (graphic display) 
and punch Ruta may drive. 

[0116] If a server 9 acquires this instruction, an instruction will be relayed, it will 
send to a punch Ruta camera, and punch Ruta will drive in a desired location. 
Thus, in order to set up the actuation target of punch Ruta on an image, it 
becomes possible to operate punch Ruta easily, without being conscious of an 
actuation instruction in a network, the delay of an image, etc. 
[0117] By computer 1, it becomes possible to display the sent panorama image 
to put on panorama actuation field 6B of a monitor 2, and to display the 
environment where the current punch Ruta camera is set up, at a glance. 
Moreover, as previous explanation also described, since the actuation range of 
the positional information of punch Ruta, a zoom lens field angle, and punch 
Ruta is displayed, it becomes possible on a panorama image to check the 
condition of current punch Ruta easily at panorama actuation field 6B. That is, if 
the instruction whose computer 1 or server 9 drives punch Ruta and a zoom is 
generated, the situation of the punch Ruta camera according to it can check on a 



panorama image. 

[0118] Moreover, on panorama actuation field 6B, when a mouse is clicked, the 
location where the mouse click of [ on a panorama image ] was carried out 
sends an instruction (absolutely location actuation instruction) to a server 9 like 1 
operation gestalt based on the positional information of the mouse when being 
clicked, so that it may come to the core of actuation field 6A (image) and may 
drive. If a server 9 acquires this instruction, an instruction will be relayed, it will 
send to a punch Ruta camera, and punch Ruta will drive in a desired location. 
Thus, in order to set up the actuation target of punch Ruta on a panorama image, 
it becomes possible to operate punch Ruta easily, without being conscious of an 
actuation instruction in a network, the delay of an image, etc. 
[0119] After all the images are connected, you may make it express it to 
panorama actuation field 6B as the operation gestalt of this invention, although a 
panorama image is connected within a computer 1 whenever an image is 
supplied to a computer 1 from the punch Ruta camera 3, and it shows the image 
to panorama actuation field 6B whenever it is connected. 

[0120] Although actuation field 6A and panorama actuation field 6B are 
expressed as the operation gestalt of this invention on the screen of the monitor 
2 connected to the computer 1 , actuation field 6A and/or panorama actuation 
field 6B may be displayed on another display which is different in a monitor 2. 



[0121] With the operation gestait of this invention, it is good also as a maximum 
range where the punch Ruta camera 3 can carry out movable [ of the range 
which can be photoed with the punch Ruta camera 3 ], and the range which can 
be photoed by the limiter may be restricted. Moreover, you may have the 
function in which the limiter restricts photographic coverage in the punch Ruta 
camera 3, and may have it in a computer 1. 

[0122] Although one point of the arbitration generated from the field of arbitration 
was set as the core of that field with the operation gestait of this invention, it is 
good also considering the circumcenter, the inner center, center of gravity, or 
orthocenter of the field of not only it but arbitration as one point of arbitration. 
[0123] Although made as [ create / by computer 1 / a panorama image ] with 
other operation gestalten of this invention, a panorama image is created by the 
server 9, the data of that panorama image are transmitted to a network circuit 
with little circuit capacity at a computer 1, and you may make it display on 
panorama actuation field 6B of a monitor 2. 

[0124] Although it considered as one computer 1 to the server 9 and the punch 
Ruta camera 3 which were installed in the remote place in order to give 
explanation easy, a server 9 and the punch Ruta camera 3 are arranged all over 
the world, for example, you may make it control one punch Ruta camera 3 by 
other operation gestalten of this invention from two or more computers through 



the Internet. 
[0125] 

[Effect of the Invention] If it depends on this invention, since the technique of 
mapping and connecting the picturized image on the virtual spherical surface is 
taken, it becomes possible to offer the image which does not have distortion in 
the connection section, a high which is obtained only using the device of high 
resolution, and a graphics format even if it connects the image of the device 
using the device of low cost, and a graphics format by this - a quality (high 
definition, high quality) - an image can be obtained. Moreover, effectiveness is 
expectable with the application which used it for the equipment (for example, 
camera with built-in punch Ruta) which can change the image pick-up direction 
of image pick-up equipment since the panorama image which generally 
becomes possible using the lens or fish-eye lens of an expensive super-wide 
angle system was created easily. 

[0126] Moreover, since a location to drive on a panorama image and a 
photographic subject to project can be specified by displaying a zoom field angle, 
a punch Ruta location, etc. on a panorama image, for example while becoming 
possible to grasp the situation of image pick-up equipment easily if it depends on 
this invention, it becomes possible although the target image is acquired easily. 
Better operability and visibility are realizable using a panorama image for such 



applications especially for control applications, such as a monitor in the video 
camera in the remote place using a circuit, observation, advice, and introduction. 



DESCRIPTION OF DRAWINGS 
[Brief Description of the Drawings] 

[Drawing 1] It is approximate line drawing showing the rough configuration of the 
system of 1 operation gestalt of this invention. 

[Drawing 2] It is approximate line drawing for explaining the screen in 1 
operation gestalt of this invention. 

[Drawing 3] It is the block diagram showing the system configuration of 1 
operation gestalt of this invention. 

[Drawing 4] It is approximate line drawing for explaining generation of the 
panorama image in 1 operation gestalt of this invention. 

[Drawing 5] It is a schematic diagram for explaining generation of the panorama 
image in 1 operation gestalt of this invention. 

[Drawing 6] It is a schematic diagram for explaining generation of the panorama 
image in 1 operation gestalt of this invention. 

[Drawing 7] It is approximate line drawing which had in the explanation which 



generates the include-angle information on a punch Ruta camera, and was from 
the position coordinate on the panorama actuation field in 1 operation gestalt of 
this invention. 

[Drawing 8] It is approximate line drawing for explaining the flat-surface 
spherical-surface conversion in 1 operation gestalt of this invention. 
[Drawing 9] It is approximate line drawing for explaining the coordinate 
transformation in the actuation field in 1 operation gestalt of this invention. 
[Drawing 10] It is approximate line drawing for explaining the coordinate 
transformation in the panorama actuation field in 1 operation gestalt of this 
invention. 

[Drawing 11] It is approximate line drawing for explaining the internal positional 
information and the include-angle information on a punch Ruta camera in 1 
operation gestalt of this invention. 

[Drawing 12] It is approximate line drawing for explaining the include-angle 
coordinate of a punch Ruta camera and the position coordinate of a panorama 
actuation field in 1 operation gestalt of this invention. 

[Drawing 13] It is approximate line drawing for explaining the field angle of the 
punch Ruta camera in 1 operation gestalt of this invention, and the frame in a 
panorama actuation field. 

[Drawing 14] It is approximate line drawing for explaining conversion of the zoom 



data in 1 operation gestalt of this invention and scale-factor data. 

[Drawing 15] It is the flow chart which shows an example of processing of the 

whole 1 operation gestalt of this invention. 

[Drawing 16] It is the flow chart which shows an example of processing of the 
panorama creation request event in 1 operation gestalt of this invention. 
[Drawing 17] It is the flow chart which shows an example of processing of the 
timer event in 1 operation gestalt of this invention. 

[Drawing 18] It is the flow chart which shows an example of the panorama 
creation processing in 1 operation gestalt of this invention. 
[Drawing 19] It is approximate line drawing showing the rough configuration of 
the system of other operation gestalten of this invention. 

[Drawing 20] It is the block diagram showing the system configuration of other 
operation gestalten of this invention. 

[Drawing 21] It is approximate line drawing for explaining creation of a panorama 
image. 

[Description of Notations] 

1 [ ... Panorama actuation field, ] ... A computer, 2 ... A monitor, 6A ... An 
actuation field, 6B 7 [ ... Punch Ruta section, ] ... Cursor, 9 ... A server, 11 ... The 
camera section, 12 13 ... TV monitor, 14 ... A pointing device, 15 ... Lens block 
section, 16 ... A zoom lens, 17 ... The zoom section, 18, 26, 27 ... Motor, 19 ... A 



solid state image pickup device, 20 ... Signal separation / automatic-gain-control 
circuit, 21 [ ... A camera controller, 25 / ... A punch Ruta controller, 28 / ... Punch 
Ruta, 29 / ... Video capture, 30 / ... The storage section, 31 / ... Control 
section ] ... An A/D converter, 22 ... A digital disposal circuit, 23 ... A mode 
controller, 24 



